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ABSTRACT 

This speech traces the inception and the operation of 
an accountability program in the Columbus, Ohio, public schools. The 
need to comply with the mandated evaluation requirement of ESEA Title 
I programs and to respond to a demand from the educational community 
that they be advised of the progress of these programs alerted the 
Columbus public schools to the fact that their system lacked proper 
performance evaluation tools. The evaluation of these programs was 
contracted, therefore, to the Ohio State University Evaluation 
Center. The speech explains how the evaluation center has 
operationalized accountability and presents the policies, processes, 
and product of the evaluation effort. (Author/JF) 
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CASE STUDY OF 

AN ACCOUNTABLE SCHOOL DISTRICT 



The cid adage about the v;eather, "Everyone talks about it, but nobody 
does anything about it," could be applied to accountability as well. Have 
you tried looking up accountability in any of the standard reference works? 
We've tried, and find the following kinds of entries; 

In the Dictionary of the S ocial Sciences , accountabi I ity i s cross- 
referenced to respons i b i I ity , where we find 

"Answerability implies (a) accountability, which focuses 
attention on the sanctions or procedures by which public 
officials may be held to account for their actions, e.g. elec- 
tora i deteat ; I eg I s I at i ve i n vest i gat i on and pub I i c i ty w i th 
attendant loss of reputation; executive removal, demotion, de- 
privation of promotion or privileges; judicial process, liabil- 
ity, etc. Other authorities emphasize (b) d i scret i onary capacity 
implicit in a public role or office which the occupant feels 
required ©r is expected fuJfill over and beyond his forma!, 
legal authority. A third denotation refers to (c) a professional. 
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group, or legal standard of performance by which the holder of 
a public trust considers himself bound, A fourth is that of 
(d) loyalty to a determinate or indeterminate, personal or in- 
stitutional, source of authority, from which emanates a material- 
ideological I nterpretation of the ends that the official structure 
of authority is supposed to serve, and to which official incum- 
bents render obedience.”* 

This defining process goes on for three more sections. It would be 
nice and scientific and scholarly to say that we started from a clearly de- 
fined concept of accountability and systematically planned and implemented 
it in operational terms and behaviors. In truth, we have not done so. 

In fact, passage and implementation of Title I, ESEA programs in early 
1966 provided one impetus for knowing whether programs were working, were 
meeting objectives. The mandated evaluation requ i rement of Title I, ESEA 
brought a realization to the Columbus Public Schools, and hundreds of other 
systems as we I I , that the capacity to perform evaluation was nonexisient 
in the school system. The evaluation of these programs was contracted to 
the Ohio State Un i vers i ty Evaluation Center. This contract was for two and 
one-haif years, and provided for program evaluation, research in evaluation, 
training of in-resident school system personnel in evaluation, and finally, 
the installation of an evaluation capacity in the school system. 

We were not using the term accountabi lity then. We were, however, 
aware that the State Department of Education had to report to the U.S. Office 
of Education on the success or failure of titled projec-fs. We, in turn, 
had to report to the State Department. At the same time, people In the 
educational community (adml n i strators , board members, teachers, parents, and 

*Juliu3 Gould and William L. Kolb, eds. , A Dictionary of . the Social 
Sciences (Mew York: The Free Press, 1964). 
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even students) began asking, ”How well are these programs doing?” As long 
as the OSU Evaluation Center was performing the evaluation, the data and 
reports, under the terms of the contract, were the property of the Columbus 
Public Schools. We were many times asked, ”How are these projects doing?” 
by the news media, by cit-zens’ groups and by classroom teachers (from whom 
we collected data) and had to respond, ”We can’t give you that information. 
See the school system about it.” Reports, of which there had been multiple 
reports prepared for distribution, disappeared, without having been dis- 
tr i buted . 

The community, particularly the inner-city, but not limited to that 
area, began increasing pressure on the schools for information about hov7 well 
students did on achievement tests. Each school in fhe system administered 
achievement tests at certain grade levels. Schools were permitted to select 
any one of three achievement batteries, or combination of subtests therein. 
Individual studfent test results were not released or shown to parents. Con- 
cern over how well the schools were doing, with the amounts being spent 
per pupil by school, with balancing school staff and student bodies racially, 
generated community pressure. This pressure was exerted on school class- 
room teachers, building principals, central office administrators and the 
Board of Education. The Board of Education was asked to have a team of 
experts, not associated with the school system, conduct, a study. Civil 
rights groups and Board members could not agree on experts, and final ly ac- 
cepted the offer of President Fawcett to provide the resources of the Ohio 
State University to make the study. This study, which was chaired by Vern 
Cunningham, was conducted from mid-March and completed by June, 1968. It 
is difficult to capture fhe flavor of a 322-page report in a few minutes. 
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However, s thread which runs through the entire report is woven of account- 
abillTy, respons i d i I i ty , and responsiveness- One might characterize the 
description of the system being studied as a "closed** system, and the recom- 
mendations made as being institutional changes toward accomplishing an open 
system. 

One chapter of this report was entitled "School System Assessment and 
Accountability." The summary of this chapter states: 

Most people are interested in how their own youngsters fare 
in school. They are likewise interested in how well their school 
system is achieving its mission. Questions about how schools 
rank in comparison to other schools too are often raised. Such 
questions are asked honestly and humbly and represent a very 
modest request. What most people do not realize, however, is 
that evaluating an institution as complex as a school system is 
a most difficult assignment. This is frue whether it is done in- 
ternally by the staff or by an outside group. 

A comprehensive evaluation of the Columbus Schools was not 
tho ass1gnw^r\+ “t*tve Cortv^iss-lon Our attention focused on 

problem areas, one of which was the need for continuing assess- 
ment of the system. Three recommendations are made in this re- 
gard. 



An Office of Evaluation and Research was authorized in 
May, 1968, by the Board of Education. The Commission is encour- 
aged to note this development and commends the board and admin- 
istration on the purposes and objectives chosen for that office. 
Our first recommendation is that the Board support this office 
generously. 

Second, we urge that school district policy on the sharing 
of test results be revised to allow for an annual report on 
school achievement. Such a report should include such Items as 
follow-up information on graduates, changes in pupil achievement, 
new types of testing that are being tried, character 1 st i cs of the 
student body being served, and where appropriate, comparisons 
with other school systems. 

Third, v-^e recommend that regional sub-district school 
assessment committees be established in 1968-69 as a part of the 
general recommendations for decentralization which are made later 
In the report. There is a genuine need in all school systems for 
Improved ways of developing community understanding about schools 
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and school understar.d i ng about communities. To repeat, many 
people want i nfomnat ion , they want to understand, they want to 
take part and, above all, they v.'ant to be confident about the 
quality of education their children are receiving. The re- 
gional assessment committees would have leading laymen, teachers, 
administrators, and students as members. They would meet each 
month and report at least annually to the Board of Education and 
to local building PTA's and other community groups.* 

What has happened since that report to make the Columbus Public Schools 
accountable? 

The authorized office for Evaluation and Research was installed as a 
department within the Division of Special Services in September, 1968. 
Virtually all of the personnel who had been trained at the OSU Evaluation 
Center were attracted to the school system, and formed the core of fhe new 
department. . These persons included professionals, secretaries and clerical 
workers (called MAT's — manual analysis technicians).- Ali but the director 
of the office and the classified office manager were supported by funds tor 
Title I evaluation and State AFDC programs. The General Fund local budget 
for the initial installation was about $25,000 on an annual basis. 

However, another recommendation of the report cited called for passage 
of an operating levy to implement program improvements, teacher salary levels 
This levy passed, and the Department of Evaluation and Research was given a 
"green light" to implement the second phase of our five-year plan. This 
phase included the recruiting and selection of professional staff to provide 
for evaluation (and research) capabilities for the system at large. Simul- 
taneously, our General Fund staff of one was developing the format of 

*Luvern L. Cunningham, et al., A Report to the Columbus Board of Educa- 
tion, "Recommendations of: The Ohio State University Advisory Commission 

on Problems Facing the Columbus Public Schools," June, 1968. 
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the first Columbus Schoo I Prof i I e , evaluating the cdvisory and assessmeni 
committees, meeting and talking with a multitude of community groups, and 
consulting with system personnel on evaluation problems. 

We received authorization from the Board of Education to employ the 
equivalent of 14 full-time professionals, and three interns, who were one- 
half time students in doctoral programs at OSU. Mosf of these persons had 
been trained and taught in the classroom, then taken advanced degree work. 

S-|-0ff members ranged from brand new bachelor^s degree teachers with under- 
graduate minors in research, master's degree candidates and Ph.D. candidates. 
Areas represented, besides educational research are: social work, sociology, 

nursing, psychology, curriculum development, information science, admin- 
istration and political science. Assimilating and molding this group of 
highly motivated persons into a team has been an exciting challenge — which 
has not ceased to be a chal lenge. 

Obviously? the personnel for the department were specifically recruited 
to meet needs for certain skills and competencies, some of which we could 
identify easily, such as design and statistics. Others we are still iden- 
tifying. As we were recruiting, the question we kept discussing was, "What 
is the role of the department?" The early, simplistic answer was, "We’re 
going to evaluate, so that the school system can be acccui.tab le." 

Accountabi I ity 

The whole question of accountability has four basic parameters: to whom, 

by whom, for what and in what way. Vie are all familiar with accountability 
for money, what I would cal I the ^*old accountabi 1 ity* 

Under the old accountability, we were accountable to the public, mors 
specifically, the taxpayer, for the right, proper, and judicious use of public 
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monies in the disciiarge of our du+ies as public educa+ors. Accoun+ab i I i 1 y » 
by whom, was legally a responsibility of the Boards of Education. Account- 
abliity, for what, was for the inputs into education, the dollars received 
from the taxpayers and expended on educational programs, buildings, equip- 
m 0 nt , materials, and personnel. Accountability, in what way, was a matter 
of budgeting, keeping account of expenditures, avoiding deficit expendi- 
tures, facilitated by a methodology of double entry bookkeeping or account- 
ing. 

Under the *'new accountabi I ity'* a new dimension has been added. No 
longer are we solely concerned with inputs. There is ample evidence that 
millions of dollars have been put into education which have had no notice- 
able impact on education. There is nothing to prevent this from continu- 
ing unless or until we become concerned with output. In the jargon of the 
military, this has been simplified into the phrase "more bang for the buck." 
i am not here to convince you that this output dimension is moi e important 
than fiscal accounting. Nevertheless, across the narion, in the legisla- 
tures, in the Stafe Departments of Education, in the communities, among 
pressure groups there is concern with the output of education. Account 
ability, the new accountability, is an accountability to not only the tax- 
payer, the public, but parents and students, as well. This accountability 
is not only by the Boards of Education, but admi n i strators and classroom 
teachers. Accountability is not just for input, in dollars, but for outpui , 
in terms of units learned, ;n terms of post school accomplishments. The 
facilitating method for this accountability is not a simple double entry 
bookkeeping system whicii keeps track of dollars, but a system v/hich keep=> 
-j-pack of output — and in ft mo, wil I have to relate input" to ouiput. 
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Accountability is like a double edged sword. That is, not ail of our 
school outputs are good, nor are they all bad. They are both positive and 
negative- It's necessary to admit the areas that need improvement, as well 
as provide information about the successes you're having. The negative re- 
ports not only help identify problem areas which generate support for needed 
change, but enhance the positive reports, make them more credible. 

A recent Sat urday Review carries a cartoon which has a subtle indica- 
tion of why accountability is necessary — A man, overhearing his wife's cof- 
fee klatsch obviously discussing their concern over sex education comments: 

What's to worry about? They 'I I teach sex like 
they do the rest of the subjects and the kids 
wi ! I ose i nterest. 

Saturday Review, 12-13-69 

The Columbus Public Schools established the Department of Evaluation 
and Research as a response to the problem of accountabi I ity. Information 
is r, significan«f part of accountability. 

Evaluation is the process of delineating, obtaining and providing use- 
ful information for judging decision alternatives.^' It's useful to remem- 
ber the acronym DOPl — for delineating, obtaining and providing information. 

Educational evaluation is being defined as a process, essenfially con- 
tinuing as long as there is something about which information is needed for 
making decisions. A process is on-going, does not happen at just one or two 
points in time. 

This process of delineating, obtam.rig and providing information is 
to reduce uncertainty, and therefore risk, in decision-making. 

I believe that this major bias of mine — What is eval uation?--that 
evaluation is so closely interactive with decision-making, has so much 

O ^Definition used by Phi Delta Kappa Monograph Committee. 
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influence on a formulation of an evaluation department that you should be 
aware of it. The professional evaluator tends to resist being the ’'tool" 
of a decision-maker. There is something nefarious, and perhaps Machiavel- 
lian about the role. However, one must reconsider the definition of decision- 
maker. 

It is essentia! to consider the term educational community . An edu- 
cational community consists of those persons in a community who have an 
interest in education — parents, teachers, students, administrators, and 
often, just interested citizens. Each of these categories represents _a 
set of educational dec i si on— makers . Parents decide to support and rein- 
force a school — or net, as the case may be. leachers make decisions about 
decisions — they may follow administrative directives or covertly ignore them. 
Students choose to learn or not learn, as reflected in their level of mo- 
tivation.' Citizens’affirm school goals, programs and progress through their 
approval of money issues at the polls. Other groups have other means of 
exp ress ing their sat i sf act i on s and diss at i sf act 1 ons w 1 i h the schoo I s . 

The evaluator is concerned with improving educational opportunities 
for children and also providing information to members of the educational 
community so that they might have an awareness and understanding of These 
educational opportunities. 

Many equate information with public relations, or -with the "trade 
puffery" familiar in advertising. The information with which the evaluator 
is concerned is not imaginary nor should it be "puffery." Anyone examin- 
ing the situation, using the same instruments, the same criteria and the 
same analyses, should be able to derive the same results. In short, 
evaluators must be credible. 
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The evaluator finds hmself as a middleman. He does not have an 
"ideal" set of criteria to use in evaluating all programs. One man's 
criterion is another's poison! He must evaluate', not to tel I an audience 
what it wants to hear, but to satisfy their need for information. Their 
need for information varies from audience to audience, dependent upon 
their value structure, their life style, their economic and educational 
concerns. Thus, the evaluator has a multivariate problem with multiple 
cr i te r i a . 

With this understanding, you can group some of the reasons for the 
charges of "whitewashing," of irrelevancy, of obfuscation. If the evalua~ 
tor determines the criteria for an evaluation, he may well be overlooking 
criteria of importance to some audience. Thus, he must attempt not only 
to Identify these audiences beforehand, but must be involved with these 
audiences so that they identify their criteria to be satisfied as well as 
become familiar with the criteria of other concerned audiences. These 
criteria may not be formally identified, but picked up unobtrusively by 
the evaluator. If a program were to be evaluated on only one criterion 
framework, such an evaluation .could we I I be disregarded by other audiences. 
It seems to be more prudent to anticipate this problem and respond to it 
in the evaluation design and report. 

A further problem lies in the level of performance or activity ac- 
ceptable to an audience. It is possible to establish a criterion level 
which is acceptable to only a small group of persons or to a limited number 
of roles In the educational community. Desired performance levels may vary 
within an audience. Rather than reporting information which results from 
applying a simple criterion level, we must attempt to provide alternatives 
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— reporting ail data so that re-analysis can be performed, reporting informa- 
tion resulting from application of several criterion levels, etc. 

The operational definition being used is: 

Educational evaluation is the process of 
delineating, obtaining and providing useful informa- 
tion for Judging decision alternatives. 

I have spoken briefly about the relationship of evaluation to decision- 
making, of the information-providing role of evaluation, and the concept of 
educational community as multiple decision-makers — and audiences for evalu- 
ative information. 

Why have ^ Department of Eva I uat i on? 

It should be evident at this point that evaluation, as a systematic, 
scientific process should be performed. It may not be as evident that 

? c I I i -ax / a I I I w "f'O tS 

- ^ cf' ‘ *7 '“.’..T * ' - — - •• _ 

able or inclined to perform the systematic evaluation which we have been 
discussing* I i" is a rare decision-maker who can d I spass i ona+e ly view a 
program or situation in which he is involved. 

The evaluator (and I use the term collectively, whether it be one or 
thirty) who works only for, or under one decision-maker has a narrow, 
limited audience, and will prepare reports for that audience which are 
likely to be as narrow or limited^ However, if the evaluator is estab- 
lished separately, if he has an identity which is not linked too closely 
to any one audience^ he wi I ! be capable of being more objective, or at 
least, broader in perspective. Evaluators must be able to relate across 
many different groups of people which may have quite divergent views. 
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Since there is a general acceptance that ”he who holds infonnation has 
power," the evaluator is both threatening and at the same time perhaps sub- 
ject to pressure- If he has a degree of autonomy and demonstrates his 
"fairness" or "objectivity" he is less likely to be threatening and less 
iikely to be pressured. He must be able to obtain information, and also 
be at a level where he has an understanding of what’s going on around him 
— "the big picture-" He should have access to the decision-making power 
structure so that he can identify information needed for those decisions- 
I earlier provided the Dictionary of Social Sciences definition of 
accountability, reviewed some of the press for accountability in the Colum- 
bus community, the recommendations of the OSU Advisory Commission for an 
"accountability arm," and the initial stages of staffing the department. 

\ talked about accountability, information, and evaluation, hopefully link- 
ing these to decision-making, all leading up to a rationale for an office 
charged with that responsibility- 

Now I’d like to make a stab at a definition of accountability: 

Accountability is the willingness to provide information 
and develop the capacity thereof by an agency on the extent 
to which the commitments of the agency are being met, and the 
responsiveness of that agency to the concerns of the educa- 
tional community in selecting, pursuing and reviewing its goals 
and objectives. 

I believe that accountability has to be a positive action rather than 
a defensive ploy. An agency, no matter how proper, must be perceived to be 
accountable- It cannot Just say, "V/e’re being accountable" unless the com- 
munity finds such a statement to be credible- The agency must not only say, 
"We’re willing," but must demonstrate its ability to be accountable, by 
recognizable action- The next stage may be even more difficult- I hear. 
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over and over again, the comment ’’You've given us all this information, 
beautiful charts, fac-s and figures, but we want to know, what are you going 
to do about it?" The second part of the definition included the phrase 
. . responsiveness of that agency to the concerns of the educational 
community in selecting, pursuing, and reviewing its goals and objectives* 
This means the agency will tune in the community, and listen to its con- 
cerns. 

I should point out that this is a working definition, derived from ex- 
perience. 



What have done to operation a I i ze accountab i I i ty? 

First, the system made a commitment to accountability by putting up 
General- Fund, local money to support the department. 

Second, . 5 he syeieni eViqbiished persoune! policies which rr-cds :t pos 
sible to hire qpmpetent personnel for non-administrative roles at competi- 
tive salaries. 

Third, the system established accountability-type policies. These 
include: 



1. A policy which requires a standardized testing program, 
comparable across schools and grade levels. 

2. A policy which requires a published annua! report' to the 
community on student performance on a school b-y school 
basis. 

3. A policy which provides parents with access to performance 
records of their child on the tests administered, and the 
student himself, as well. 

4. A policy which established survey committees, consisting of 
elected students, community people, parents and faculty mem 
bers, at each school, whose function Is to serve in an ad- 
visory capacity, to facilitate communication between school 
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administration and students, teachers, parents and citi- 
zens. Meetings are open to other students, teachers and 
citizens to bring their concerns to the committee. 

5. A policy calling for follow-up of graduate careers, to 
include data on school performance and post high school 
activity, as well as feedback from the graduates themselves. 

The seeds of these policies can be fou^^',’ in the basic recommendations 
of the OSU Advisory Commission. On the whole, the school system could be 
said to have met these recommendations well. I ^d like to mention some of 
the other areas in which we^ve been working. 

Student unrest resulted in some unfortunate incidents in the Spring of 
1969 in several of our s"=^condary schools. In one building, many students 
engaged in a sit-down strike, were hauled off to jail. This action re- 
sulted in reaction, and it was clear that a plan to cope with student un — 
r«si wa^ iitjeued. As iiie plan cJeveiopau io hanuie s i uden i unresi, ii was 
clear that we should also plan to prevent student unrest. The resulting 
student relations policy is two-fold: the first section deals with things 

the school must do to provide for student concerns, to assess student atti- 
tudes, to provide legitimate channels of communication for students, to 
Insure broad representat i on of the student body in student government, etc. 
The second part of the policy is to prevent escalation of student unrest, 
if the first part of the policy fails; finally, the policy provides for 
moderate hand I ing of eruptions which have not been prevented. Note that 
building admi n istrators, faculties, parents and students all have responsi- 
bilities and rights which are being recognized. Also note the building 
admi n i strato r is accountable for having taken the preventative measures. 

Another step in accountability was taken last year when the Board of 
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Education agreed in professional negotiations with the Columbus Education 
Association to an annual building evaluation, to be conducted by faculties, 
on a form to be jointly developed by the Board and the CEA. This evaluation 
was to cover professional environment, staff support, student relations, 
and co-curricular activities. Our office developed the instrument used, 
aided in its administration, and are now developing a system-wide report 
for the Board. The faculties each reported to their principal and the CEA 
their findings. I'd note parenthetically that we made a lot of friends with 
teachers during that experience, but alienated a lot of building prin- 
cipals. 

We are now in the process of reciprocating. The Board and the CEA have 
agreed to adopt a new evaluation system of teachers, which we have been 
given the responsibility for developing. This is partially in response to 
' demands from the community that teachers be accountable, and partially in 

response to the need for information to make decisions about re-hiring, ter- 
minating and promoting to tenured status. 

Our system is plagued with over-crowding, a defeated bu i I d i ng propose I 
and an interest in the space-saving economics attributed to the Year Around 
School. As part of the accountability policy, we are now engaged in a 
comprehensive study of the Year Around School , on v^hich the Board wi I I base 
thei r deci sion. 

Accountability extends to all Board of Education meetings. Committee of 
the Whole meetings and sub-committee meetings being open to the public and 
the media. We are all learning how to do business in public. 

Much of what has been said relates to total system activities and 
policies. I’d like to deal for a while with the kind of policies which 
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permit a capability for accountability to exist. 



Pol icies 

First, and perhaps the most important, is the need to be completely 
open and honest with the department. They must know what's going on so 
that they can anticipate information needs. The department must also be 
credible to al I its audiences, all those members of the educational com- 
munity. This demands an openness of reporting or the right to publish 
evaluation findings in their unexpurgated form--both good and bad, while 
at the same time, sensitizing administrators to the findings so that they 
can provide a well thought out response. 

Second, the department must have unlimited access to data in the system 
with the understanding that individuai data, such as reports on teachers 
and siuuenis wi j i noi* be revealed publicly. 

Third, th^ department must be involved in setting its priorities, but 
these priorities must be congruent to the goals of the system. 

Fourth, the department must have budgetary support for systematic evalu 
ation and research. Up to I per cent of total system budget has been recom- 
mended, similar to R & D budgets of industry^ Separate budgets for mandated 
funded projects (Title I) may run 5-7 per cent of the budget. Our office 
never got to the I per cent level. We were operating at about a five-tenths 
of one per cent level. However, the system is now in a state of budget 
deficit, so we lost the equivalent of 6 1/2 professionals from our General 
Fund operations. 

Keep in mind that evaluation costs are also opportunity costs. You 
may save thousands of dollars over several years by having evaluative 
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information about programs. You may avoid costly program investments with 
low payoffs, or be able to terminate or modify such programs before they 
become costly mistakes. Or you may be able to maintain the public^s con- 
fidence that a course of action being followed is the best of available 
strategies . 

Fifth, the department must have competent personnel. It is important 
to recruit, train and retain the personnel necessary to perform the task 
at hand. I would like to make a case for evaluation, or accountability 
personnel, as being different. They* re not classroom teachers on special 
assignment, they’re not guidance counselors doing program evaluation, they’re 
not principals who have retired from the battleground. They are competent 
specialists, rather than genera I i sts. -They have probably received some 
specialized training, or should receive such training, to. meet the informa- 
tion providing needs of their position. Universities should be doing more to 
train evaluation personnel to work in the field. More evaluators are now 
being trained in the field, on the. job, than in our graduate schools. This 
could easily be a collaborative venture of an educational administration 
faculty, a research and evaluation faculty, and the public schools near the 
universities. 

It is essential to foster collaborative relationships between the 
universities and public school systems. There are evaLuation activities and 
developmental tasks which are beyond the means of either institution operat- 
ing solo.. There is a need for on the Job, .field training of evaluators, 
through a device such as we have used — the internship. They receive, in 
addition to their formal classroom training, on the job training, a reality 
orientation; and we receive a source of personnel, a recruiting mechanism. 
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and a major source of ideas. I regret to say that the budget cuts have re- 
duced our Internships to zero, with no restoration in sight. 

School systems must consider other pol i ci es -for evaluation to be suc- 
cessful and for accountability to work effectively and efficiently. One 
of these deals with the utilization of extant data wherever possible. This 
means generating a standardized data base that can be counted on and called 
upon, that data can be retrieved from, effortlessly and painlessly. 

The evaluator must have access to information in the system if the de- 
cision-maker is to utilize the data in making decisions. How often is data 

ignored if it is not readily available, or available only after the fact? 

Some data can be used, over and over again, for many different purposes. For 
example, we have computerized the beginnings of a student file, which In- 
cludes standardized test information. This data is used for preparing in- 
dividual pupil i<"ofiles of test results, and made available to teachers, 
counselors, pri'ncipals, parents— and even the students. We have found ways 
to present the data so that the old business of releasing a point estimate 
of a test result is no longer a problem. We use the same master file to 
summarize information for each classroom and for buildings. We are ready 
to provide, on an optional basis, an item analysis of standardized test re- 
sults to classroom teachers, and to curriculum specialists, as well as 
scoring and returning teacher fiiade tests along with such an analysis. 

We use the same data for the School Profi le , the building by building 
summarization of test results .and other factors pub I ished and made freely 
available to the community. 

Finally, the same data is utilized in context evaluation for identi- 
fying problem areas, used as a basis for proposal writing for Title I projects. 
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and for allocating resources within the school system. 

The evaluator not only has to have access to the information in the 
system and the mechanisms for creating data, but he must have access to 
the decision-making structure. He must interact with the decision-makers 
sn delineating information needs. He must help the decision-maker anti- 
cipate what information will be needed in the future. 

Since the evaluator is dealing with information which may b.e negative 
to the maintenance portion of the system, he must have a measure of autonomy 
from the maintenance parts of the organization — for protection, as wel I as 
to ma i nta ? n cred i b i I i ty for accountab i I i ty . My co I I eagues i n the system 
may not always appreciate this, but to have evaluation reports labeled as 
credible is a very important part of the strategy of accountability. The 
day when major sectors of the educational community mistrust or disbelieve 
accounts from our office, I have lost my usefulness and will have to be 
replaced by someone who is credible. 

Does ^ Department of Eva! uation 
do eve rythi ng re I ated to i nformat ion? 

Our definition of educationai evaluation — the process of delineating, 
obtaining and providing useful information for judging decision alterna- 
tives— this definition might lead one to believe that evaluation and the 
Department of Evaluation is attempting to deal with, and in fact control, 
all the information a decision-maker uses. Nothing could be further from 
the truth. Educational evaluation is, in fact, just one more way of know- 
ing. To be of utility, educational evaluation must provide information 
which. is not available by other means, cannot be obtained at lower cost 
otherwise, and is useful in the judging of decisior alternatives. While 
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evalua+ors may a+ times sound like the answer to all our problems, we are 

not and indeed can do no more than help in the solution of problems toward 

the improvement of education. 

The Department of Evaluation and Research has both a 

Direct Services Role 
and a 

Facilitating Services Role 

which provides the system with: a direct capacity to perform evaluation and 

research, and a facility to help others in the system, such as 

building p ri nci pa I s 
facu I ty 

curriculum specialists 
supervi sors 

project directors to evaluate 

The department, in essence, holds "the Goodhousekeeping" Seal of Approval 
for evaluations — to ensure that the accountability is credible, reliable, 
valid. 



Processes 

Our department is concerned with bui Iding mechanisms into the system 

to provide a priori evaluations wherever possible 

Computer based student file 
Computer based teacher file 

In-service training for professional staff 

Dissemination 

Feedback of reports to sources: audiences 

- Item analysis to classroom teachers and 

building principals 

- Information to community on neighborhood seminars 

- School profile — individual school profiles — 

printed 200,000, taken home by every child 

Establishing survey committees in every school, which are 
both data sources, and information users. 
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Interfaces 

with the educational community 
Administrators 

- Parents 

- Pub lie 

- Students 

- Teachers 

Many evaluation activities are of an ad hoc or special purpose nature. 

The damand for our direct services in evaluation far exceeds our capacity 
and will continue to do so. Recognizing this, we have established several 
developmental task areas which aro intended to extend the capacity of our 
department to serve the information needs of the school system. We are 
attempting to utilize the concept used by financial institutions and holding 
companies which is called I eve rage . Our intent is to increase the evalua~ 
tion capacity of the system without a directly proportionate increase in 

We r'hr>ccin- +o do t*^ I s ass'"^i'^g a model sca'~ce 

resources; among others funding is limited, personnel are limited, and space 

* 

is limited. Either one sets priorities and only attacks prob.lems v/ith the 
highest priority, or uses leverage or a combination of both. Our concept 
of leverage works like this: many evaluations. Title I, State DP, the 

School Profile , etc., require the same or similar data. Much of this data 
is already being created and collected for some purpose in the system already. 
We are working on a systematic identification of extant data which our 
office can utilize. This, of course, is one of the first steps in a systems 
analysis. We have developed a standardized testing program to Tit the needs 
of teachers and counselors and also provide our office with approximately 
75 per cent of our testing requirements for Title I through this step alone. 

Having some direct involvement with program evaluation, such as Title I, 
provides us with a training and testing ground for evaluation personnel 
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and techniques. Once tried and modified as necessary, a technique can then 
be used by other persons in the school system with the knowledge that it 
will work. Thus, we are able to provide consultants on evaluation to the 
school system at large. These field team consultants have the responsi- 
bility for helping personnel in the field, such as principals, teachers, 
curriculum supervisors, etc., with evaluation designs, instrumentation, 
data reduction, analysis and reporting. However, the responsibility for 
conducting these ad hoc: evaluations will remain with the school. We are 
hopeful that our field consultants will also be training persons in the 
many different offices of the school system, through an ”on the job" train- 
ing experience, so that, when they are through, they have a greater under- 
standing of, appreciation for, and usage of evaluation. We are trying to 
work ourselves out of business by training others to "do it themselves." 

Products . • 

Typical products of our department include: 

The Columbus School Prof I le 

The Columbus Testing Profile and its data by products 

The Student Opinion Survey 

The Building Evaluation System 

The Neighborhood Seminar Reports 

The Graduate Fol Icw-Up Study 

Voter intention Surveys 

Special Program Evaluation Reports 

Mandated Evaluation Reports 
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Phenomena 

Results of School Prof i le and other activities: 

The large problems (system-wide) are broken into smaller 
ones (building levei). 

The credibility of the school system is improving. 

Decisions are more informed — 

Neighborhood Seminar recommendations 
Survey Committees in secondary schools 
Voter Intention Surveys 

Higher degrees of data utilization — not just flowing from 
bottom up, but flowing back to the source of information. 

Removal of myths, assumptions, for example, the transfer 
and assignment of teachers. 

Dealing with substantive questions, not with questions of 
accountab i I i ty . 

The Department of Evaluation and Research has a tenuous, shifting base 
of support. We are hot winning popularity contests. Theoretically, we should 
be able to convince everyone that we are there to help them do their job more 
effectively. Practically speaking, we threaten them. 

We have to look at the success of the school system over the past two 
years in beginning to be accountable, and not suffering cis much from being 
accountable as when we were not being accountable. 

We have to look at the last school year and the significant improvement 
in the degree of student unrest. 

We have to look at the total educational community and its beginning 
, efforts to work with the school system to improve educational opportunities. 

We find in these things some very real hope that accountability will 
v/ork, the school system vn 1 I improve and the community will benefit. 
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